Error Correlation and Error Reduction in Ensemble Classifiers

Using an ensemble of classi ers, instead of a single classi er, can lead to improved generalization. The gains obtained by combining however, are often a ected more by the selection of what is presented to the combiner, than by the actual combining method that is chosen. In this paper we focus on data selection and classi er training methods, in order to \prepare" classi ers for combining. We review a combining framework for classi cation problems that quanti es the need for reducing the correlation among individual classi ers. Then, we discuss several methods that make the classi ers in an ensemble more complementary. Experimental results are provided to illustrate the bene ts and pitfalls of reducing the correlation among classi ers, especially when the training data is in limited supply.